RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights #400

Gnimuc · 2018-12-03T12:42:31Z

fix #388, @Roger-luo you might want to give it a test ;)

It looks that `dump` is no longer exposed in LLVM release build.

Roger-luo · 2018-12-14T19:04:42Z

Still get this.

 Building Cxx → `~/.julia/packages/Cxx/2NIIy/deps/build.log`
┌ Error: Error building `Cxx`:
│ [ Info: Building julia binary build
│ ERROR: LoadError: UndefVarError: JULIA_HOME not defined
│ Stacktrace:
│  [1] top-level scope at logging.jl:312
│  [2] top-level scope at /Users/roger/.julia/packages/Cxx/2NIIy/deps/build.jl:43
│  [3] include at ./boot.jl:317 [inlined]
│  [4] include_relative(::Module, ::String) at ./loading.jl:1041
│  [5] include(::Module, ::String) at ./sysimg.jl:29
│  [6] include(::String) at ./client.jl:388
│  [7] top-level scope at none:0
│ in expression starting at /Users/roger/.julia/packages/Cxx/2NIIy/deps/build.jl:42
│ writing path.jl file
│ Tuning for julia installation at /Users/roger/.virtualenvs/julia/bin with sources possibly at /Users/roger/.virtualenvs/julia/bin/../..
└ @ Pkg.Operations ~/Documents/Repos/julia/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1069

Gnimuc · 2018-12-15T02:00:49Z

@Roger-luo it looks like you were doing a binary build, but this PR is for fixing source build glitches(e.g. REPL mode). For the binary build, please check out this
#390 (comment). With this PR, just unzip the tarball to the deps folder and run using Cxx.

Roger-luo · 2018-12-16T21:20:19Z

I see. This means the binary will be provided separately in the future?

Gnimuc · 2018-12-17T01:18:05Z

yep, just like other BB2 projects.

fix-up

Roger-luo · 2019-03-05T11:33:38Z

Seems fine. But segmentfaults when

using Cxx

C++> #include<#iostream>
true

C++> std::cout << "aa";

Gnimuc · 2019-03-05T11:37:48Z

@Roger-luo are you working on a source-build or binary-build?

This PR should work well for the source-build on MacOS. For binary-build, please take a look at those comments in #390. It looks like @cdsousa is working on a script for better user installation experience: #390 (comment)

BTW, make sure you've done a cleanup on Cxx.jl's dep folder before trying any of these methods. I suspect the segment faults are triggered by an old installation.

Roger-luo · 2019-03-05T13:33:49Z

I built Julia from source, with MKL.

Gnimuc · 2019-03-05T13:38:54Z

debug build or release build?

Roger-luo · 2019-03-05T13:45:57Z

release build, I'm rebuilding now, deleted deps/build

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.2.0)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libimf
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom

Roger-luo · 2019-03-05T14:02:24Z

I deleted build and rebuilt the whole thing, now I get

julia> using Cxx

C++ > #include<iostream>
true

C++ > std::cout << "aaa" ;

signal (11): Segmentation fault: 11
in expression starting at no file:0
_ZNK4llvm18IntrusiveRefCntPtrIN5clang11FileManagerEEdeEv at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
_ZNK5clang16CompilerInstance14getFileManagerEv at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
EnterSourceFile at /Users/roger/.julia/dev/Cxx/deps/usr/lib/libcxxffi.dylib (unknown line)
unknown function (ip: 0x12b1a8fc8)
unknown function (ip: 0x12b1a8f4c)
macro expansion at /Users/roger/.julia/dev/Cxx/src/cxxstr.jl:701 [inlined]
cxxstr_impl at /Users/roger/.julia/dev/Cxx/src/cxxstr.jl:701
unknown function (ip: 0x12b1a8f2c)
jl_fptr_trampoline at /Users/roger/Documents/Repos/julia/src/gf.c:1864
isTopLevelExpression at /Users/roger/.julia/dev/Cxx/src/CxxREPL/replpane.jl:103
#1 at /Users/roger/.julia/dev/Cxx/src/CxxREPL/replpane.jl:166
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
do_respond at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:707
unknown function (ip: 0x12b19abd4)
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
run_interface at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/LineEdit.jl:2273
run_frontend at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:1035
run_repl at /Users/roger/Documents/Repos/julia/usr/share/julia/stdlib/v1.1/REPL/src/REPL.jl:192
#734 at ./client.jl:362
jfptr_#734_6051 at /Users/roger/.virtualenvs/julia/lib/julia/sys.dylib (unknown line)
jl_apply at /Users/roger/Documents/Repos/julia/src/./julia.h:1571 [inlined]
jl_f__apply at /Users/roger/Documents/Repos/julia/src/builtins.c:556
jl_f__apply_latest at /Users/roger/Documents/Repos/julia/src/builtins.c:594
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
run_main_repl at ./client.jl:346
exec_options at ./client.jl:284
_start at ./client.jl:436
true_main at /Users/roger/.virtualenvs/julia/bin/julia (unknown line)
main at /Users/roger/.virtualenvs/julia/bin/julia (unknown line)
Allocations: 11817660 (Pool: 11815841; Big: 1819); GC: 26
[1]    75462 segmentation fault  julia --color=yes

any idea?

Gnimuc · 2019-03-05T14:29:01Z

OK, I can reproduce the segmentfault as well with a fresh build. It looks like Cxx failed to init file manager (those two symbol demangled to llvm::IntrusiveRefCntPtr<clang::FileManager>::operator*() const and clang::CompilerInstance::getFileManager() const).

This is probably due to some API changes on the LLVM side(Julia1.0 use LLVM 6.0.0, but Julia1.1 use LLVM 6.0.1), which is not compatible with the current implementation of libcxxffi. (I haven't dived into the code, so this is just a wild guess.)

~~Welp, Cxx.jl source-build is broken again. :( A workaround is to use binary-build if you do wanna use this package. The libcxxffi binary for Julia 1.0 is compatible with Julia 1.1.~~

UPDATE: all tests can actually pass without any problem, the segmentation fault is REPL-only.

Gnimuc · 2019-03-05T15:17:02Z

@Roger-luo the real reason is that the newer GC caught an upgrading oversight.

Roger-luo · 2019-03-05T17:12:14Z

Ok, this is fixed now. Maybe we don't need 0.7 in the CI now? since it's not supported anymore. And you need to use cp -XR on MacOS in the .travis config, MacOS has some extended attributes that you have to ignore.

Roger-luo · 2019-03-05T17:31:10Z

Could you fix the travis configuration here?

      if [ ! -f $HOME/early_abort ]; then
          julia --project=. -e 'Pkg.test(coverage=true)' || false
      fi

should be

      if [ ! -f $HOME/early_abort ]; then
          julia --project=. -e 'using Pkg; Pkg.test(coverage=true)' || false
      fi

Roger-luo · 2019-03-05T17:35:25Z

I think this PR should at least pass linux and OS X CI now (tests are passed on my laptop), and maybe someone with access could review this and merge it into master, then tag a new release. (at least to let people try and find out bugs) @Keno

Gnimuc · 2019-03-05T23:54:49Z

X-ref: #373

Gnimuc · 2019-03-09T15:40:40Z

@Roger-luo see also #406
CI for Linux is green: https://travis-ci.org/Gnimuc/Cxx.jl

Keno · 2019-03-09T17:00:06Z

I'm unlikely to have much time to spend on this in the near future, but I'm happy to add people to the repo for commit access and do point reviews as necessary. Does that sound like a plan?

Keno · 2019-03-09T17:01:33Z

src/cxxstr.jl

-    C = instance(CT)
-    id = sourceid(sourcebuf)
-    buf, filename, line, col, disable_ac = sourcebuffers[id]
+    GC.@preserve CT begin


Why is this required. CT should be a type and not GC eligible.

Roger-luo · 2019-03-09T17:03:40Z

Yes! That would be really nice. @Keno I think a lot people is trying to contribute here.

Keno · 2019-03-09T17:07:24Z

Alright, transferred to JuliaInterop. Have fun.

Gnimuc · 2019-04-12T14:13:46Z

closed in favor of #406

Gnimuc added 5 commits December 3, 2018 20:39

dump() => print(llvm::errs(), false)

63d3a42

It looks that `dump` is no longer exposed in LLVM release build.

contains => occursin, a upgrading oversight

9d71f9f

fix-up

aaaba51

Fix REPL mode

2292c5c

Misc. upgrading oversight

3fff0d4

Gnimuc mentioned this pull request Dec 7, 2018

Make Cxx.jl ready for production environments post Julia 1.0.0 #390

Closed

fix-up

613430f

Gnimuc mentioned this pull request Dec 9, 2018

post v1.0 support issue adding package #397

Closed

Update cxxstr.jl

20643ae

Gnimuc mentioned this pull request Jan 5, 2019

Fix some v1.0 incompatibilities in show.jl. #401

Merged

Gnimuc changed the title ~~Fix source build on macOS~~ Fix source build(macOS), REPL mode and misc. upgrading oversights Jan 5, 2019

cdsousa and others added 2 commits January 25, 2019 11:10

fix-up

2e8c5b2

Merge pull request #1 from cdsousa/patch-mac

2c7641b

fix-up

Gnimuc mentioned this pull request Mar 5, 2019

ERROR: LoadError: failed process on MacOS Julia v1.1 #404

Closed

Gnimuc changed the title ~~Fix source build(macOS), REPL mode and misc. upgrading oversights~~ Fix source build(macOS), REPL mode and misc. upgrading oversights(Julia 1.0) Mar 5, 2019

Fix segmentfault

94071c4

Gnimuc changed the title ~~Fix source build(macOS), REPL mode and misc. upgrading oversights(Julia 1.0)~~ RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights Mar 5, 2019

Keno reviewed Mar 9, 2019

View reviewed changes

Remove meaningless GC preserve

9cc8ab4

Gnimuc closed this Apr 12, 2019

Gnimuc deleted the patch-mac branch April 12, 2019 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights #400

RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights #400

Gnimuc commented Dec 3, 2018 •

edited

Loading

Roger-luo commented Dec 14, 2018

Gnimuc commented Dec 15, 2018 •

edited

Loading

Roger-luo commented Dec 16, 2018

Gnimuc commented Dec 17, 2018

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019

Roger-luo commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019 •

edited

Loading

Gnimuc commented Mar 5, 2019

Roger-luo commented Mar 5, 2019

Roger-luo commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019 •

edited

Loading

Gnimuc commented Mar 5, 2019

Gnimuc commented Mar 9, 2019

Keno commented Mar 9, 2019

Keno Mar 9, 2019

Roger-luo commented Mar 9, 2019

Keno commented Mar 9, 2019

Gnimuc commented Apr 12, 2019

RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights #400

RFC: Fix source build(macOS), REPL mode and misc. upgrading oversights #400

Conversation

Gnimuc commented Dec 3, 2018 • edited Loading

Roger-luo commented Dec 14, 2018

Gnimuc commented Dec 15, 2018 • edited Loading

Roger-luo commented Dec 16, 2018

Gnimuc commented Dec 17, 2018

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019 • edited Loading

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019

Roger-luo commented Mar 5, 2019 • edited Loading

Roger-luo commented Mar 5, 2019

Gnimuc commented Mar 5, 2019 • edited Loading

Gnimuc commented Mar 5, 2019

Roger-luo commented Mar 5, 2019

Roger-luo commented Mar 5, 2019 • edited Loading

Roger-luo commented Mar 5, 2019 • edited Loading

Gnimuc commented Mar 5, 2019

Gnimuc commented Mar 9, 2019

Keno commented Mar 9, 2019

Keno Mar 9, 2019

Choose a reason for hiding this comment

Roger-luo commented Mar 9, 2019

Keno commented Mar 9, 2019

Gnimuc commented Apr 12, 2019

Gnimuc commented Dec 3, 2018 •

edited

Loading

Gnimuc commented Dec 15, 2018 •

edited

Loading

Gnimuc commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019 •

edited

Loading

Gnimuc commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019 •

edited

Loading

Roger-luo commented Mar 5, 2019 •

edited

Loading